Multiplication of conjugacy classes, colligations, 
and characteristic functions of matrix argument 

YURY A. NERETir\0 

We extend the classical construction of operator colligations and characteristic 
functions. Consider the group G of finite block unitary matrices of size a + oo + ■ • • + oo 
(k times). Consider the subgroup K = U(oo), which consists of block diagonal unitary 
matrices (with a block 1 of size a and a matrix u g U(oo) repeated k times). It appears 
that there is a natural multiplication on the conjugacy classes G//K. We construct 
'spectral data' of conjugacy classes, which visualize the multiplication and are sufficient 
for a reconstruction of a conjugacy class. 



1 Characteristic functions 

1.1. Notation. By 1 = 1„ we denote the unit matrix of order n. Let Mat(n) 
be the space of complex matrices of size n x n, GL(n, C) the group of invertible 
matrices of order n, U(n) be the unitary subgroup in GL(n, C). By GL(oo, C) 
we denote the group of finite invertible matrices of size oo, i.e. infinite matrices 
g such that g — 1 has only finite number of non-zero elements. By U(oo) we 
denote the subgroup in GL(oo,C) consisting of unitary matrices. Denote by 
Mat(oo) the space of all matrices, that are finite in the same sens^. 

For a group G and a subgroup K we denote by G//K the conjugacy classes 
of G with respect to K. 

1.2. Structure of the paper. This section contains preliminaries (results 
of [13]): a construction of 'spectral data', and main Theorems ll.10iH.lll Proofs 
are contained in Section 2. 

Product of conjugacy classes. Fix integers a ^ and m ^ 1. 



1.3. 

Let N > 0. Consider the space Mat(ev 
matrices 
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mN). We write its elements as block 



dim 



(1.1) 



of size a + N + ■ 
following matrix 
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N. For an element u £ GL(7V, C) we denote by l(u) the 
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We also will use the notation 



Denote by 



a b \ }a 
c d) }mN 



M N = M% m :=Mat(a + mN)//GL(N,C) 



the set of conjugacy classes of Mat(a + mN) by GL(iV, C), i.e., the quotient 
with respect to the equivalence relation 



i{u)gi(u) 1 . 



(1.3) 



We call elements of this set by colligations. 

Usually, we omit superscripts a, m from M^ m . 

Next, we formulate the definition of a colligation in another form. Consider 
the space V = C m and Zn = <C N . Then an operator g can be regarded as an 
operator 

c a ® (v o z N ) -> c a e (v ® Zjv) 

defined up to a conjugation by an element of GL(N, C) acting in Z^v- 
We wish to define a canonical multiplication 
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(a, m are fixed). First, we consider the case m = 2. Then the multiplication is 
given by 
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and we take the conjugacy class containing the latter matrix. 

A passage from m = 2 to arbitrary m is evident. However we formulate 
a formal definition, which is valid for all m. We extend the operator g to an 
operator g in 



c a e (v ® z Nl+N2 ) 



(v®z Nl ) 



v®z 



A' 2 
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This operator acts as g on the first summand and as 1 on the second summand. 
Next, we extend h to an operator h acting in the same way in the space 



c Q e (v® z Nl+N2 ) = c a ®(v®z N2 ) 



v®z Nl 



and set 



g o h := g • h. 



Proposition 1.1 The o -multiplication is associative, i.e., for any N±, N%, N3 

and gi 6 M.Nu 92 G M.n 2 ; 93 G -Mn 3 the following equality holds: 

(91 °92) °33 = 5i {92093)- 
This is more or less obvious (see also }13|). 

1.4. Infinite-dimensional version. Denote by Moo the set of conjugacy 
classes on Matoo with respect to GL(oo,C). Then Moo is a semigroup with 
respect to o-multiplication. 

1.5. Variants. Denote by QCn conjugacy classes of GL(a + mN,C) with 
respect to GL(7V,C), by Un the conjugacy classes of U(q + mN) with respect 
to U(iV); in all cases we consider the equivalence (| 1 . 3[) . in the unitary case we 
suppose that u is unitary. Thus we get the operations 

GC-oo x G£oo — > G£oc', 

hi 00 x Uoo Uoo- 

1.6. The origin of o-multiplication. The spaces A^^ 1 and U^ 1 (and 
some similar objects) are classical topics of the spectral theory of non-self-adjoint 
operators and system theory (see e.g., [10], [11], [2], [3], [20], [6], [4], Chapter 
17). In 80s a family of operations of this type appeared in representation theory 
of infinite-dimensional groups, see [16], [12], Section IX. 4. For instance, there is 
a semigroup structure on double cosets 

O(oo) \U(a + oc)/0(oo). 

Moreover this semigroup acts in O(oo)-fixed vectors of unitary representations 
of U(a + 00). Big zoo of operations on K \ G/K (with 'small' subgroups K) 
arose in [13], [TJ]; our U(a + moo)//U(oo) is inside this zoo, 

G//K = diag(JC) \ (G x K)/&mg{K ) 

For another explanation, see [15] (the o-product is the limit of convolutions of 
^-measures on conjugacy classes at infinite-dimensional limit). 

The purpose of this work is to obtain spectral data of matrices visualizing 
the o- multiplications for m > 1. 
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Notice that the spectral data for collections of matrices were widely dis- 
cussed, see different approaches in surveys p], [8]. It seems that our approach 
produces another kind of spectral data. On the other hand, it extends classical 
approach of operator theory. 

1.7. Categorical quotient. Notice that the spaces Ai jy are non-Hausdorff. 
There are many ways to construct Hausdorff spaces from set-theoretical quo- 
tients. We prefer the following. 

Let a reductive group G act on an affine algebraic variety X. Consider the 
algebra C(X) of G-invariant regular functions on X. The categorical quotient 
[X/G] is the set of maximal ideals of C(X) G . 

We consider categorical quotients [A4n] of Mat(a + mN) by GL(iV, C) and 
[GC N ] of GL(a + mN, C) by GL(7V, C). 

Notice that regular functions on Mat (a + miV) are polynomials. The algebra 
of regular functions on GJj(a+mN, C) is generated by polynomials and det(<?) -1 . 

Proposition 1.2 a) The natural map [Mn] ~~ ^ [^n+i] is injective. 

b) The natural map [GCn] — > [G£n+i] is injective for N > ma. Moreover, 
for N > ma we have [QCn] — [M.n]- 

c) For N > ma the natural map Ujq — >• Un+i is injective. 

The statement is proved in Subsection 12.41 

This allows to define the infinite-dimensional 'categorical quotient' [.Moo] as 
the inductive limit. On the other hand this shows that the topological quotient 
Uoo is Hausdorff. 

Proposition 1.3 The o -multiplication determines a map of categorical quo- 
tients, 

The statement is proved in Subsection 12.51 

1.8. Characteristic functions. Let g £ Mat(TV). Let S range in Mat(m). 
We write the following equation: 
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Eliminating variables x\, . . . , x m , we get a dependence 

q = Xg(S)p, 

where Xg(S) ^ s a rational matrix- valued function, Mat(m) — > Mat(a). We call 
X g by the characteristic function of g. 

The following statements were obtained in [13] . 
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Proposition 1.4 The characteristic function of g depends only on a GL(JV,C)- 
conjugacy class of g (and not on g itself). 



Proposition 1.5 Consider the natural map In : M.n 

Xl N g{S) = Xg{S)- 

This is obvious. 



M 



N+l- 



Then 



Theorem 1.6 



Xgoh{S) = Xg{S)Xh{S). 



Theorem 1.7 Let g be unitary. Then 

a) 7/||S[| < 1, then \\ X g{S)\\ < 1. 

b) If S is unitary, then Xg(^) * s unitary. 

Remark. Thus we get a multivariate analog of inner functions (see, e.g. 

0, QS1). 

Denoting S :— S ® ljv, we represent (jl.5p as 

q = ap + bSx 
x = cp + dSx 



□ 



Therefore 
and 

Proposition 1.8 



x = (1 — dS) 1 cp 
X 9 (S)=a + bS{l-dS)- 1 c. 



dot 



detx 9 (S) 



a —bS 
\c 1 - dS 

det(l - dS) 



(1.6) 



To observe this, we apply the formula for a determinant of block matrix to 
the numerator of (11.61). 



1.9. The language of Grassmannians. Now we reformulate the defini- 
tion of characteristic function. For a linear space W denote by Gi(W) the set 
of all subspaces in W. For an even-dimensional space denote by Gr 1/2 (W) the 
set of subspaces having dimension ^ dim W. 

Consider the Grassmannian Gr 1 ^ 2 (y © V) in the linear space V (B V. For 
any L C Gt 1/2 (V © V) consider the subspace L® Z N dV ® Z N . We write the 
equation 
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Consider the set X g (L) of all (q,p) such that there are (x, y) £ L®Z^ satisfying 
(|1.7[) . The set X g (L) is a subspace in C a © C". The following statement is 
straightforward . 

Proposition 1.9 Let L be a graph of an operator S : V — > V . Assume that S 
is a non- singular point ofxg- Then X g (L) is the graph of the operator X g {S) : 

Thus X g is a rational map Gr 1/2 (V © V) -> Gr 1/2 (C a © C a ). 

Remark. Notice that the map Gr 1/2 (V®V) -> Gr(C a ©C") is well-defined 
on the whole Gr 1/2 (V®V). □ 

1.10. Distinguished divisors. A characteristic function does not de- 
termines an element of A4n- Indeed, consider the following matrix of size 
a+ (k + l) + (k + l) 
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Then its characteristic functions does not depend on the matrix 

Therefore additional invariants are necessary. 
Let g £ Mat(n). We write the equation 
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and consider the set £ ff of all S such that this equation has a nonzero solution. 
Clearly, all poles of x§ are contained in £ g . 

It is more convenient to reformulate the definition in a more precise form. 
Consider the polynomial p g on Mat(m) determined by the equation 



Pg (S) = det(l - dS) = 

lmJV " 
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Denote by P g (S) the denominator of det Xg(S) (we assume that the fraction 
is reduced). Bv ll.61 P g (S) is a divisor of p g (S). Denote 



n g {S) :-- 



Pg(S) 
Pg(SY 
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The set £ ff is the set of zeros of the polynomial p g . Moreover, it is more 
natural to consider the set £ s as a divisor (see, e.g. [7]), i.e. we decompose 
p g {S) as a product of irreducible factors 



Pg 



(S)=Y[h t (Sr, 



where hi are pairwise distinct. Then we consider the collection of hypersurface^f] 
hi(S) — with assigned multiplicities Vi. 
Evidently, 

Pgoh(S) =p g {S)p h (S). 

Equivalently, 

^goh — £,g 

The sign '+' means that we consider the union of hypersurfaces taking in ac- 
counts their multiplicities. 

Let us pass to the language of Grassmannians. Consider another matrix 
coordinate A = S^ 1 on Grassmannian. Then the equation for the divisor passes 
to the form 
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det(l - dS) = 0, det(d - A) = 

determine a divisor E g in Gr 1 / 2 (V©V). Indeed, we have two charts in Gr 1/,2 (Vffi 
V), one consists of graphs of operators V © — > © V, another of graphs of 
operators © V — > V © 0. These charts do not cover the whole Grassmannian, 
but the complement of their union has codimension 2. Therefore any hypersur- 
face in Gr 1 ' 2 (V r © V) has an intersection with at least one of charts. In fact 
all hypersurfaces in Gr 1 ^ 2 (y © V) are observable in the chart S, except the 
hypersurface det A = (it is a complement to the chart S) . 

The degree of the polynomial p g for generic g is mN . If the divisor 2 g 
contains the component det A = with multiplicity I, then the degree of p g is 
m(N - 1). 

1.11. Characteristic functions for A4oq. We have a natural embedding 

I N : M N -> M N +i 
induced by the embedding Zn — > Zn+i- Evidently, 

Xl N g(S) = Xg(S)- 



3 of complex codimension 1. 
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Therefore, we can define a characteristic function for elements of = Un-Mn- 
On the other hand, we can repeat the definition of characteristic function for 
N = oo, this produces the same result. 
Next, 

Pi Ng (S)= Pg (S)-det(l-S). 
Denoting by 6 the divisor det(l — S) = we get 

= Zg+8. 

Thus we can define a 'divisor' E g C Gr^fV ffi V) for g e .Moo. Notice that its 
component 8 has multiplicity oo. Except 6, we have finite number of components 
of finite multiplicity. 

Remark. In the same way, £i Ng = £ g + 8. We can define £ s for g 6 Moo. 
However, acting in this way we loss information concerning the multiplicity of 
the divisor det A = 0. □ 

1.12. Central extension. Denote by T the multiplicative semigroup of all 
rational functions x '■ Mat (to) — > Mat (a). Denote by A the Abelian additive 
group of all divisors in Gr 1 / 2 (F © V). We send 

However, the image of this map does not split to a product. The reason is 
Proposition II .81 Indeed, let pass to the data (x g (S), n g (S)). Then 

n g oh(S) = K g (S)n h (S) ■ c 9i h(S), 

where 



P 9 (S)P h (S) 



Pgoh(S) 

is a nontrivial cocycle (it is responsible for cancellations of poles of Xg(S) an d 
zeros of Xh{S) in the product Xgoh(S) = Xg(S)Xh(S). 

1.13. More invariants. Let j = 1, 2, For g G GL(oo, C) we consider 

the direct sum 

g l3] := g ffi • ■ • © g 
of j copies of g. It acts in the space 

c a ffi (v ® z N )} ffi • • • ffi [c Q ffi (v ® z N )] = a a © ((v © • • • © v) ® z N ) = 

= <c ja ®{{v ®<c J )® z N ). 

Thus we get an embedding 

it is compatible with o-multiplication: 

ff M o/jb] = (goh) lj] . 
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Now for a given g 6 M°^ m we get the collection of characteristic functions 
Xg, X g m , X g m , • • • ; X g m : Mat(jm) Mat(ja) (1.9) 
and the collection of divisors 

E g ,Z gl2h Z gl3h ...; Z gl]] cGY((V®a) ® (V®a)). (1.10) 

Theorem 1.10 a) For an?/ a, m, N the characteristic functions (|1.9[) and di- 
visors (jl.101) uniquely determine an element of the categorical quotient [M^ ]. 

b) for anj/ iV > am t/ie characteristic functions (|1.9I) and divisors (| 1 . 10[) 
uniquely determine an element of the categorical quotient [QC^J 771 ]. 

Proof is contained in Subsections I2.1H2.2I 



Theorem 1.11 a) For any N > am the characteristic functions (jl.9p and 
divisors (|1 . 10[) uniquely determine an element of the space Up m of conjugacy 
classes. 

b) The characteristic functions (ll.9[) and divisors (11.101) uniquely determine 
an element of the space U^ m of conjugacy classes. 



Proof is contained in Subsection [ 
1.14. Relations between characteristic functions of different levels. 

Proposition 1.12 a) Let Si, S2 be matrices of sizes mji, mj'2 respectively. 
Then 

XgUx+n] ((^q = Xgini(Si) ®x g m(S2)- (1.11) 

b) For an invertible matrix H of size j denote by H the operator 
H = H®lj : V (g> CP V (g> C J 



inV <g> . Then 

c) The divisor E 9 m is invariant with respect to the transformations of 



X g[j] (HSH- 1 ) = H Xq[j] (S)H- 1 . 



Gr 1/2 ((l/®C 3 )©(T/Cg)C 3 )). 



induced by transformations H © H . 
d) Consider the natural embedding 



Gr 1/2 ((V ® C J1 ) © (V ® C J1 )) x Gr 1/2 ((V O 2 ) © (V ® C 32 )) 

-> Gr 1/2 ((F ® C J ' 1+J2 ) © (V C J ' 1+J ' 2 )). 

(/or a pair of subspaces L\, L2 we assign their direct sum Li © L2 in the target 
space) . We have the following coincidence of divisors 

EgUi+M f] [Gr 1/2 ((V ® C 31 ) © (V © C n )) x Gr 1/2 ((V ® C-' 2 ) © (V ® C-' 2 )) 
= H flUll xGr 1/2 ((F®C j2 )ffi(F®0 2 )) + Gr 1/2 ((T/®C J1 )©(^®C J1 )) xS sJ2l 
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2 Invariants 



2.1. Invariants for .A/Q m . Consider the matrix (II. ip . Denote 

a [il = a (g> lj, = 6 (g) 1 J; c [iI = e<g> L,-, = <2<g> L,-. 

Denote by 6/3 [A:] the fc-th row of the matrix bp, by c 7 [i] the Z-th column of the 
matrix c 7 . Let us regard Ma,t(a+mN) as a linear space with action of GL(N, C). 
A point of the space is a collection of m 2 matrices , am vectors c 7 [I] , and 
am of covectors bp [k]. Generators of the algebra of invariants are known (see, 
e.g., [19], Section 11.8.1). The algebra is generated by the following polynomials 

tr d^ 1 ip 1 dip2Tp2 ' ' " ^<PnlpTi > (2T) 

bp[k], (2.2) 
a aT . (2.3) 

We wish to show that all the generators can be expressed in terms of Taylor 
coefficients of x g u\ (S) and p g ui (S) at zero. 
First, consider the expression 

lnp gW (S) = lndct(l - dPS), (2.4) 

where S is an operator in V ® . 

It is convenient to think that matrix elements of S depend on 4 indexes: 
indices tp, tp ^ m are responsible for an operator in the space V ~ C™ 1 , and /1, 
v ^ j for operators in the space C 7 '. In a neighborhood of S 1 = we have the 
following expansion 



lndetfl - d [j] S) = V ^ '- tr (d^SY 



n>0 



(-1)" 



n ^ 0, 

ipt,...,tp n ,ipt,...,ip n < "I, 

X tr d^, 1 ,(p2^T/ l 2 ,^3 " " ' ^"fpn^l ' (2"5) 

Generally, Taylor coefficients (in s^) of this series are sums of traces. However, 
assume that all elements of the collection /xi, . . . , /i„ are pairwise distinct. Then 
the coefficient at 



IS 



We observe that all the invariants (|2.ip are contained in the set of Taylor coef- 
ficients of (l2~4l). 
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Next we consider the characteristic function 



XgW (S) = a [jl + b^S(l - d^S)- 1 ^ . 
Substituting S = 0, we get invariants of the type (|2.3|) . Let us expand 

oo 

X := b [l] S(l - d^S)- 1 ^ = b m S(dWS) n C W (2.6) 

n=0 



in a Taylor series. The operator X acts in C m ® . We enumerate its matrix 

7(5 
C fcZ 



elements as xlf, where k, I ^ a and 7, <5 ^ j. The matrix elements are 



71 > 

(pi, . . .,¥?„+! < m 

■ ■ ■ ,1pn+l < "1 

x b l p 1 [k]dip 1 ^ l p 2 di{ ;2 <p 3 . . . G^> n) ^3 n _|_ 1 c^> n _|_ 1 [7] 
Again, consider a term of the Taylor series at 

>Pl,4>l <P2,^2 ' ' ' Pn+liti'n+l 

where /i2, ■ • ■ , /in are pairwise distinct and do not equal 7, 5. The factors d... 
in the coefficient and their order are are uniquely determined. 
This proves Theorem II .lOl a. 

2.2. Invariants for invertible matrices. Now consider the space QL°^ m '. 
Invariant regular functions on this space have the form u(g)/ det(g) k , where 
p(g) is a polynomial satisfying 

u(i,(u)fft(u) _1 ) = det(g) k u(g), where u € GL(7V, C), 

i(u) is given by (|1.2[) . Therefore, u(g) is invariant with respect to SL(N, C). 
In comparison with (I2.1j) - (|2.3[) we get new invariants (see [TS], Section 11.8.1). 
First, consider an iV-plet of pairwise distinct vectors bp [k], compose a matrix 
with such rows. Its determinant is an invariant of SL(7V, C). In the same way we 
compose determinants from columns c 7 [7]. These determinants are additional 
invariants. 

However, we can do this only if N ^ am (the total number of possible rows). 
Thus for N > am there arc no additional invariants. 
This proves Theorem ll.lOt b. 

2.3. Invariants for unitary group. Consider invariants of U(JV) on 
U(a + mN). Consider the algebra A of functions on U(n) generated by matrix 
elements and det(g) -1 . By the Peter-Weyl theorem it is dense in the algebra of 
continuous functions. 
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Consider two orbits 0\, O2 C U(a + mN) of U(iV), consider an invariant 
continuous function ip separating these orbits. Consider a function ip G A 
approximating tp, say 



We have 



^ a < b < tb 



*G0 = Pi9) 



(det. 9 ) fe ' 

where p is polynomial. The average 

/(<?)= / ^(t^giihr^dh 

J\J(N) 

has the same form o|t^p w ith polynomial g. Therefore / has a regular contin- 
uation to the whole group GL(a + mN, C). If TV > am, then all such functions 
are polynomials in Taylor coefficients of Xg(S) and p g (S). 



This proves Theorem II .111 



2.4. Change of AT. Next, we wish to show that the map In '■ [M"' n ' 



[M^^il is an embedding (Proposition [h2]) . The map In replaces 

a — >• a 

b p ^(b fj 0); 





dipip 

1 



Restrict generators (|2~T|) - (f2~3l) of invariants of g e Mat(a+m(A r +l)) to Mat(a- 
mN). We get the same expressions in all the cases except traces of the form 



tr d^p-^ d^ 2 U32 * * * ^fn^Pr) 



they are shifted by 1. Therefore, restricting a collection of generators of GL(iV+ 
1, C)-invariants of Mat(a + mN) we get a collection of generators for algebra of 
GL(N, (C)-invariants on Mat (a + mN). 
Thus we proved the desired statement. 



2.5. Categorical quotient. Proposition 11.31 follows from Theorem 11.101 
(invariants of o-products are uniquely determined by invariants of factors). 

2.6. Proof of Proposition 11.121 The statement a) is obvious, d) is 
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straightforward. To derive the rest, we set j = 2. The statement c) follows from 



«*(•«- (o2)fe 2))- 

= dctfw - ft "V!' 11 Mft" M~'Y 

^ V V V^l ^22/ V S 21 S 22/ \/l21 ^22/ ^ 

For simplicity we write hij instead of hij ■ l m <S> ljv and instead of ® ljv- 
But d commutes with scalars hij and we get c). 

Next, we write the value of the characteristic function at HSH -1 , 

fa 0\fb (A //m h 12 \(su s 12 \(hu h^y 1 x 

\0 ^0 6/ ^21 ^22/ \ S 21 S 22/ \/l21 h 22 J 

x ( f 1 °\ - f d °\ f hi1 hi A f sn si A ( hu hi2 V 1 ^ 

[\0 l) \0 d) \h 2l h 22 ) \s 21 s 22 ) \h 21 h 22 J J 
We keep in mind that hij are scalars, for this reason 

(b 0\//iu-l m jv h\ 2 ■ lmjvA _ (hu ■ l a h 12 -l a \(b 0\ 
\0 b)\h 21 -l mN h 22 -l mN J \h 21 ■ l a h 22 -l a J\0 b) 1 

Applying this several times we get 




'h n ■ 1 
Mi ■ 1 




d 
d 
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